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Abstract 

Pursuant  to  the  Department  of  the  Treasury  s  fiscal  year  2002  appropriations  act,the  Internal  Revenue 
Service  (IRS),in  November  2001,  submitted  to  the  congressional  appropriations  committees  its  fifth 
expenditure  plan,requesting  $391  million  from  its  Business  Systems  Modernization  (BSM)fund.l  As 
required  by  the  act,we  reviewed  the  plan.  Our  objectives  were  to  (l)determine  whether  the  plan  satisfied 
the  conditions  specified  in  the  act,2  (2)determine  IRS  s  progress  in  implementing  modernization 
management  controls  and  capabilities,and  (3)provide  any  other  observations  about  the  plan  and  IRS  s 
BSM  program. 


Subject  Terms 


Number  of  Pages 

90 


GAO 

_ Accountability*  Integrity  ★  Reliability 

Highlights 


Why  GAO  Did  This  Study 

The  Internal  Revenue  Service 
(IRS)  has  underway  a  multiyear, 
multibillion-dollar  business 
systems  modernization  program. 
Since  1995,  GAO  has  designated 
this  program  high-risk,  in  part 
because  of  its  size  and 
complexity  and  its  immense 
importance  to  improving  IRS 
mission  performance  and 
accountability.  The  Congress  has 
mandated  that  GAO  review  IRS’s 
plans  for  spending  fimds  on  this 
program.  This  study  responds  to 
that  mandate  by  providing  the 
results  of  GAO’s  review  of  IRS’s 
November  2001  expenditure  plan, 


What  GAO  Recommends 

To  address  the  escalating  risks 
facing  IRS  in  this  critical  program, 
GAO  recommends  that  the  IRS 
commissioner  reconsider  the 
planned  scope  and  pace  of  the 
program  as  defined  in  the 
November  2001  expenditure  plan. 
This  is  necessary  to  better 
balance  the  number  of  systems 
acquisition  projects  underway  and 
planned  with  IRS’s  capacity  to 
manage  this  workload.  GAO  is 
providing  specific 
recommendations  to  the 
commissioner  on  how  to  strike 
such  a  balance.  In  response,  the 
commissioner  agreed  with  GAO’s 
recommendations,  and  described 
planned  and  ongoing  efforts  to 
address  each. 
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Highlights  of  GAO-02-356,  a  report  to  the  Subcommittee  on  Treasury  and  General  Government, 
Senate  Committee  on  Appropriations  and  the  Subcommittee  on  Treasury,  Postal  Service  and  General 


What  GAO  Found 

IRS’s  November  2001  expenditure  plan  defines  the  next  incremental  set 
of  management  and  systems  acquisition  activities  that  are  intended  to 
move  IRS  closer  to  its  systems  modernization.  Related  to  the  activities 
proposed  in  this  plan,  IRS  has  to  date  made  important  progress  in 
estabhshing  the  systems  infrastructure  needed  to  allow  the  introduction 
of  future  business  application  systems.  Similar  progress  has  also  been 
made  in  establishing  the  key  modernization  management  controls 
needed  to  achieve  the  kind  of  systems  acquisition  activities  provided  for 
in  this  plan.  Nevertheless,  as  IRS  moves  forward  to  implement  the  plan,  it 
faces  increasing  risk  that  it  will  be  unable  to  deliver  promised  system 
capabilities  on  time  and  within  budget.  Reasons  for  this  include: 


•  IRS  lacks  certain  modernization  management  controls  and 
capabilities.  For  example,  it  lacks  a  human  capital  management 
strategy  for  the  program  and  a  reliable  project  cost-estimating 
process.  It  has  also  yet  to  implement  mature  systems  and  software 
acquisition  process  controls;  its  plans  for  correcting  this  allow  core 
modernization  projects  to  continue  for  at  least  2  more  years  without 
such  controls. 

•  Interdependencies  among  ongoing  systems  acquisition  projects  and 
the  complexity  of  associated  activities  to  be  performed  during  the 
period  covered  by  the  plan  are  an  order  of  magnitude  greater  than 
those  of  the  previous  plan.  This  workload  increase  raises  concern, 
because  IRS  was  not  able  to  meet  many  of  the  project  cost  and 
schedule  commitments  made  in  the  less  demanding  prior  plan. 

•  IRS’s  plan  provides  for  increasing  its  workload  by  starting  additional 
projects  that  will  further  tax  existing  capability. 


In  attempting  to  balance  the  need  for  introducing  modern  systems  and 
their  associated  benefits  as  soon  as  possible,  with  the  need  for  greater 
management  capacity  to  ensure  that  these  systems  are  introduced 
successfully,  IRS’s  November  2001  plan  emphasizes  the  former.  This 


emphasis  adc 


s  significant  risk  that  escalates  as  the  program  advances. 


Project  W  orkload 


This  is  a  test  for  developing  highlights  for  a  GAO  report.  The  full  report,  including  GAO's  objectives,  scope,  methodology,  and  analysis,  is  available 
at  www.gao.gov/cgi-bin/getrpt7GAO-02-356.  For  additional  information  about  the  report,  contact  Randolph  C.  Hite  (202-612-3439).  To  provide  comments 
on  this  test  highlights,  contact  Keith  Fultz  (202-612-3200)  or  E-mail  HighlightsTest@gao.gov. 
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1 

GAO 

^ _ 

United  States  General  Accounting  Office 
Washington,  D.C.  20548 


February  28,  2002 

The  Honorable  Byron  L.  Dorgan 
Chairman 

The  Honorable  Ben  Nighthorse  Campbell 
Ranking  Minority  Member 

Subcommittee  on  Treasury  and  General  Government 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Ernest  J.  Istook,  Jr. 

Chairman 

The  Honorable  Steny  H.  Hoyer 
Ranking  Minority  Member 
Subcommittee  on  Treasury,  Postal  Service 
and  General  Government 
Committee  on  Appropriations 
House  of  Representatives 

Pursuant  to  the  Department  of  the  Treasury’s  fiscal  year  2002 
appropriations  act,  the  Internal  Revenue  Service  (IRS),  in  November  2001, 
submitted  to  the  congressional  appropriations  committees  its  fifth 
expenditure  plan,  requesting  $391  million  from  its  Business  Systems 
Modernization  (BSM)  fund.^  As  required  by  the  act,  we  reviewed  the  plan. 
Our  objectives  were  to  (1)  determine  whether  the  plan  satisfied  the 
conditions  specified  in  the  act,^  (2)  determine  IRS’s  progress  in 
implementing  modernization  management  controls  and  capabilities,  and 
(3)  provide  any  other  observations  about  the  plan  and  IRS’s  BSM  program. 

On  December  10  and  11,  2001,  we  briefed  your  respective  offices  on  the 
results  of  our  review.  This  report  transmits  the  materials  used  at  those 
briefings,  and  reiterates  the  recommendations  to  the  commissioner  of 
internal  revenue  that  we  specified  in  the  briefings.  The  full  briefing 


'  The  Treasury  and  General  Government  Appropriations  Act,  2002  (P.  L.  107-67). 

^  The  act  specifies  that  BSM  funds  are  unavailable  until  IRS  submits  to  congressional 
appropriations  committees  for  approval  a  modernization  expenditure  plan  that  (1)  meets 
the  Office  of  Management  and  Budget’s  (OMB)  capital  planning  and  information  technology 
investment  control  review  requirements;  (2)  complies  with  IRS’s  enterprise  architecture;  (3) 
meets  IRS’s  life-cycle  management  requirements;  (4)  is  approved  by  IRS,  Treasury,  and 
OMB;  (5)  is  reviewed  by  GAO;  and  (6)  complies  with  federal  acquisition  requirements  and 
management  practices. 
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materials,  including  our  scope  and  methodology,  are  reprinted  in  appendix 

1.  In  summary,  we  made  the  following  five  major  points: 

•  IRS’s  November  2001  expenditure  plan  satisfied  the  conditions  specified 
in  the  appropriations  act.  However,  while  the  plan  provided  for 
satisfying  these  conditions,  IRS  must  still  fully  implement  the  controls 
and  capabilities  described  in  the  plan. 

•  Since  our  June  2001  report,®  IRS  has  made  important  progress  in 
implementing  modernization  management  controls  and  capabilities  and 
addressing  our  past  recommendations.  Nevertheless,  IRS’s 
modernization  management  capacity  is  still  not  where  it  needs  to  be, 
given  (1)  the  number  of  systems  acquisition  projects  that  the  November 
2001  plan  identifies  as  being  underway,  (2)  the  fact  that  several  of  these 
ongoing  projects  have  already  entered  the  critical  building  stage  of  their 
life  cycles  (milestone  3)  and  are  to  begin  deployment  (milestone  4) 
during  this  year,  and  (3)  IRS’s  plan  to  begin  additional  projects. 
Examples  of  modernization  management  controls  and  capabilities  that 
are  not  yet  fully  implemented  include  software  acquisition 
management,^  configuration  management,®  quality  assurance,  risk 
management,  enterprise  architecture®  implementation,  human  capital 
management,  integrated  program  scheduling,  and  cost  and  schedule 
estimating. 

•  The  increased  risk  of  IRS’s  proceeding  without  these  controls  and 
capabilities  has  contributed  to  actual  project  cost,  schedule,  and 
performance  shortfalls.  For  example,  in  the  fifth  plan,  IRS  reports  that 
the  Customer  Account  Data  Engine's  (release  1)  milestone  4  date  has 
been  delayed  by  6  months,  and  its  cost  has  increased  by  $5  million  (13 


^  U.S.  General  Accounting  Office,  Business  Systems  Modernization:  Results  of  Review  of 
IRS’s  March  2001  Expenditure  Plan,  GAO-01-716  (Washington,  D.C.:  June  29,  2001). 

^  Carnegie  Mellon  University’s  Software  Engineering  Institute  has  developed  criteria,  known 
as  the  Software  Acquisition  Capability  Maturity  Model®  (SA-CMM®),  for  determining 
organizations’  software  acquisition  management  effectiveness  or  maturity.  Capability 
Maturity  Model  and  CMM  are  registered  in  the  U.S.  Patent  and  'Trademark  Office. 

®  Configuration  management  is  the  means  for  ensuring  the  integrity  and  consistency  of 
systems  modernization  programs  and  project  products  throughout  their  life  cycles. 

Through  effective  configuration  management,  for  example,  integration  among  related 
projects  and  alignment  between  projects  and  the  enterprise  architecture  can  be  achieved. 

®  An  enterprise  architecture  is  an  institutional  blueprint  defining  how  an  enterprise  operates 
today,  in  both  business  and  technological  terms,  and  how  it  wants  to  operate  in  the  future. 
It  also  includes  a  roadmap  for  transitioning  between  these  environments. 
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percent  above  the  fourth  plan's  funding  level).  Also,  since  submitting 
the  fifth  plan  for  approval,  IRS  announced  that  the  Security  and 
Technology  Infrastructure  Release's  milestone  4  date  has  been  delayed 
by  3  months,  and  its  cost  has  increased  by  $6  million  (24  percent  above 
the  fourth  plan’s  funding  level). 

•  IRS  acknowledges  the  need  to  strengthen  its  modernization 
management  controls,  and  recognizes  that  these  controls  become  more 
critical  as  the  size  and  complexity  of  the  BSM  program  continues  to 
increase.  It  also  has  actions  underway  to  fully  implement  these  controls 
and,  until  then,  plans  to  compensate  for  their  immaturity  by  applying 
experienced  human  capital. 

•  In  our  view,  reliance  on  a  combination  of  existing  immature  processes 
and  individual  expertise  and  heroic  efforts  is  a  short-term  solution  to  a 
long-term  need.  Given  that  the  immaturity  of  these  controls  has  already 
contributed  to  project  cost,  schedule,  and  performance  shortfalls,  and 
will  likely  continue  to  do  so,  IRS  needs  a  better  strategy  for  mitigating 
the  risks  it  faces  in  implementing  its  fifth  expenditure  plan.  To  assist 
IRS  in  striking  a  proper  balance  between  the  need  to  quickly  introduce 
modernized  systems  yet  prudently  manage  the  risks  inherent  in  such  an 
undertaking,  we  made  the  following  recommendations  to  the 
commissioner  of  internal  revenue. 


Recommendations 
Executive  Action 


for  To  address  the  escalating  risks  facing  IRS  on  its  BSM  program,  we 

recommend  that  the  commissioner  of  internal  revenue  reconsider  the 
planned  scope  and  pace  of  the  BSM  program  as  defined  in  the  fifth 
expenditure  plan,  with  the  goal  of  better  balancing  the  number  of  systems 
acquisition  projects  underway  and  planned  with  IRS’s  capacity  to  manage 
this  workload.  At  a  minimum,  the  commissioner’s  reconsideration  should 
include 


•  slowing  ongoing  projects  and  delaying  new  project  starts  to  reduce 
Business  Systems  Modernization  Office  resource  demands, 

•  making  correcting  modernization  management  weaknesses  a  top 
priority  and  a  matter  of  top  management  attention,  and 

•  reapplying  resources — financial  and  human  capital — available  from 
slowed  and  delayed  projects  toward  correction  of  control  weaknesses. 

We  further  recommend  that  the  commissioner  take  the  following  actions 
with  respect  to  each  of  the  modernization  management  weaknesses  that 
we  identified. 
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First,  for  software  acquisition  management, 

•  immediately  assess  critical  BSM  projects  (i.e..  Customer  Account  Data 
Engine,  Security  and  Technology  Infrastructure  Release,  and  e-Services) 
against  the  Software  Engineering  Institute’s  Software  Acquisition 
Capability  Maturity  Model®  (SA-CMM®)  level  2  requirements; 

•  based  on  this  assessment,  develop  a  plan  for  correcting  identified 
weaknesses  for  these  projects,  including  having  an  independent  SA- 
CMM  evaluation  performed  on  them  before  submission  of  the  next  BSM 
expenditure  plan; 

•  submit,  with  the  next  expenditure  plan,  the  results  of  this  independent 
evaluation,  along  with  a  plan  for  ensuring  that  all  BSM  projects  that  have 
passed  milestone  3  will  meet  SA-CMM  level  2  requirements;  and 

•  require  all  projects  that  did  not  pass  milestone  3  as  of  December  31, 
2001,  to  be  assessed  as  SA-CMM  level  2,  and  have  a  plan  for  correcting 
any  project  weaknesses  found  as  a  condition  of  milestone  3  approval. 

Second,  for  configuration  management,  risk  management,  enterprise 
architecture  implementation,  human  capital  strategic  management, 
integrated  program  scheduling,  and  cost  and  schedule  estimating,  ensure 
that  commitments  discussed  herein  for  addressing  residual  weaknesses  are 
implemented  as  planned,  and  report  any  deviations  from  these  planned 
commitments  to  IRS’s  appropriations  subcommittees. 

Third,  until  contractor  quality  assurance  weaknesses  are  corrected, 
increase  the  level  of  IRS  oversight,  scrutiny,  and  quality  assurance  of 
contractor  activities. 

Finally,  to  allow  for  effective  congressional  oversight  of  the  program,  we 
reiterate  our  prior  recommendation  that  the  commissioner  report  to  IRS’s 
appropriations  subcommittees  any  changes  to  expenditure  plan 
commitments  concerning  systems  requirements/capabilities  to  be 
delivered  and  the  associated  benefits  to  be  realized,  and  continue  to  report 
such  performance  measures  in  future  expenditure  plans. 


Agency  Comments 


In  commenting  on  a  draft  of  this  report,  the  commissioner  of  internal 
revenue  agreed  with  our  recommendations,  adding  that  IRS  leadership 
understands  the  importance  of  correcting  the  issues  that  we  identified  and 
is  moving  aggressively  to  resolve  them.  In  this  regard,  the  commissioner 
described  IRS’s  ongoing  and  planned  efforts  relating  to  addressing  each  of 
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our  recommendations.  The  commissioner’s  written  comments  are 
reprinted  in  appendix  II. 


We  are  sending  copies  of  this  report  to  the  chairmen  and  ranking  minority 
members  of  other  Senate  and  House  committees  and  subcommittees  that 
have  appropriations,  authorization,  and  oversight  responsibilities  for  the 
Internal  Revenue  Service.  We  are  also  sending  copies  to  the  commissioner 
of  internal  revenue,  the  secretary  of  the  treasury,  the  chairman  of  the  IRS 
oversight  board,  and  the  director  of  the  Office  of  Management  and  Budget. 
Copies  are  also  available  at  our  Web  site  at  www.gao.gov. 

Should  you  or  your  offices  have  questions  on  matters  discussed  in  this 
report,  please  contact  me  at  (202)  512-3439.  I  can  also  be  reached  by  e-mail 
at  hiter@gao.gov.  Key  contributors  to  this  report  are  listed  in  appendix  III. 


Randolph  C.  Hite 

Director,  Information  Technology  Architecture 
and  Systems  Issues 
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Appendix  I 

Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 


Results  of  Review  of  IRS’  November  2001  Business 
Systems  Modernization  Expenditure  Plan 


Briefing  to  the  Staffs  of 
the  Senate  Committee  on  Appropriations, 
Subcommittee  on  Treasury  and  General  Government 

(on  December  1 1 , 2001) 
and 

the  House  Committee  on  Appropriations, 
Subcommittee  on  Treasury,  Postal  Service, 
and  General  Government 
(on  December  10,  2001) 
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Appendix  I 

Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 


GAO 
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Introduction 


•  As  mandated  by  IRS’  FY  2002  appropriations  act,  Business  Systems 
Modernization  (BSM)  funds  are  unavailable  until  IRS  submits  to  the 
congressional  appropriations  committees  for  approval,  a  modernization 
expenditure  plan  that: 

•  Meets  the  Office  of  Management  and  Budget’s  (0MB)  capital 
planning  and  information  technology  (IT)  investment  control  review 
requirements; 

•  Complies  with  IRS’  enterprise  architecture;'' 

•  Meets  IRS  life  cycle  management  requirements;^ 

•  Is  approved  by  IRS,  Treasury,  and  0MB; 

•  Is  reviewed  by  GAO;  and 

•  Complies  with  federal  acquisition  requirements  and  management 
practices. 


•  Since  mid-1999,  IRS  has  submitted  a  series  of  expenditure  or 
“spending”  plans  requesting  release  of  BSM  appropriated  funds. 


■■An  Enterprise  Architecture  (EA)  is  an  institutional  blueprint  defining  how  an  enterprise  operates  today,  in  both  business  and  technology  terms, 
and  how  it  wants  to  operate  at  some  point  in  the  future.  An  EA  also  includes  a  roadmap  for  transitioning  between  these  environments. 

2|RS  refers  to  its  life  cycle  management  program  as  the  Enterprise  Life  Cycle  (ELC),  which  is  graphically  depicted  in  the  Background  Section. 
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Appendix  I 

Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 
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Introduction 


•  To  date,  about  $968  million  has  been  appropriated  for  BSM,  including 
about  $391  million  for  FY  2002.  Of  the  $968,  $577  million  has  been 
released;  about  $391  million  remains. 


•  On  November  13,  2001 ,  IRS  submitted  its  fifth  plan,  seeking  release  of 
the  remaining  $391  million. 


•  If  the  plan  is  approved,  the  BSM  fund  will  have  a  zero  balance.  To 
replenish  the  fund,  IRS  plans  to  request  $450  million  via  its  FY  2003 
budget  request. 
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Appendix  I 

Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 
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Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 
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Objectives 


•  As  agreed  with  IRS’  appropriations  subcommittees,  our  objectives  were 
to 

•  determine  whether  the  fifth  expenditure  plan  satisfies  the  legislative 
conditions, 

•  determine  what  progress  IRS  has  made  in  implementing 
modernization  management  controls  and  capabilities,  and 

•  provide  any  other  observations  about  the  fifth  plan  and  IRS’  BSM 
program. 
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GAO 

Accountability  *  Integrity  *  Reliability 


Scope  and  Methodology 


•  To  accomplish  our  objectives,  we 

•  Reviewed  the  fifth  expenditure  plan  and  met  with  IRS  program 
officials  to  understand  the  scope  and  content  of  the  plan; 

•  Analyzed  the  plan  against  the  legislative  conditions  to  identify  any 
variances; 

•  Reviewed  program  and  project  management  reports  and  briefings 
to  assess  progress  in  implementing  modernization  management 
controls  and  capabilities; 

•  Observed  modernization  executive  steering  committee  and 
subcommittee  meetings  to,  among  other  things,  document  how  the 
plan  was  developed  and  reviewed; 

•  Interviewed  program  and  project  management  officials  to 
corroborate  our  understanding  of  the  plan  and  other  BSM  activities. 
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Accountability  *  Integrity  *  Reliability 


Scope  and  Methodology 


•  Analyzed  available  evidence  on  recent  efforts  to  implement 

modernization  management  controls  and  capabilities.  Specifically, 
we  analyzed  progress  and  plans  for 

•  enterprise  architecture  (EA)  definition  and  implementation, 

•  ELC  definition  and  implementation,  including  configuration 
management,  quality  assurance  and  risk  management, 

•  software  acquisition  maturity,  as  defined  by  the  Software 
Engineering  Institute’s  (SEI)  Software  Acquisition  Capability 
Maturity  Model™(SA-CMM), 

•  human  capital  management, 

•  cost  and  schedule  estimating  practices, 

•  integrated  program  schedule  development,  and 

•  key  projects,  such  as  the  Security  and  Technology  Infrastructure 
Release  (STIR),  the  Customer  Account  Data  Engine  (CADE), 
e-Services,  and  the  Internet  Refund  and  Fact  of  Filing  project. 
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Accountability  *  Integrity  *  Reliability 


Scope  and  Methodology 


•  Collaborated  with  the  Treasury  Inspector  General  for  Tax 
Administration  (TIGTA)  to  avoid  duplication  of  effort  in  reviewing 
BSM  initiatives  and  incorporated  TIGTA  results  in  this  briefing  where 
appropriate. 


•  Projects  addressed  by  TIGTA  included  Customer 
Communications,  e-Services,  the  Telecommunications 
Enterprise  Strategic  Program,  and  Customer  Relationship 
Management-Exam. 


•  Program-level  processes  addressed  in  a  recent  capping  report 
on  key  system  development  processes  (e.g.  requirements 
management,  configuration  management,  risk  management). 
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GAO 

Accountability  *  Integrity  *  Reliability 


Scope  and  Methodology 


•  As  agreed  with  your  offices,  we  did  not  independently  validate  planned 
initiatives’  cost  estimates  or  confirm,  through  system  and  project 
management  documentation,  the  validity  of  IRS-provided  information  on 
the  initiatives’  content  and  progress. 

•  We  provided  a  draft  of  this  briefing  on  December  7,  2001 ,  to  IRS  BSM 
program  executives  (Deputy  Associate  Commissioners  for  Program 
Management  and  Systems  Integration  and  the  Executive  Program 
Advisor  for  Risk  Management),  and  have  incorporated  their  comments 
where  appropriate. 


•  We  performed  our  work  from  October  through  December  2001  in 
accordance  with  generally  accepted  government  auditing  standards. 
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A  G  A  0 

Background 

Accountability  *  Integrity  *  Reliability 

Summary  of  IRS’  Fifth  Expenditure  Plan  ($000)3 

Program  Management  and  Architecture  Activities 

Program  Management  Office 

$  7,918 

Business  Transformation  Planning 

$  10,461 

Architecture  &  Systems  Integration 

$  32,539 

Quality  Management  and  Assurance 

$  10,082 

Federally  Funded  Research  and  Development  Center  -  MITRE 

$  18.070 

Subtotal 

$  79,070 

Project  Level  and  Infrastructure  Activities'^* 

Core  Infrastructure  Support  Projects  (e.g.  STIR) 

$107,959 

Business  Systems  Support  Projects  (  Customer  Account  Data  Engine, 

$112,224 

Custodial  Accounting  Project,  Core  Financial  Systems) 

Business  Systems  Projects 

$  89.686 

Subtotal 

$309,869 

Addition  to  Management  Reserve 

$  2,061 

Total 

$391,000 

Source:  IRS 

^See  appendix  1  for  a  more  detailed  summary  of  the  plan. 

“^The  3  categories  under  this  heading  include  16  separate  projects. 
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GAO 

Accountability  *  Integrity  *  Reliability 


Background 


•  IRS’  plan  is  to  (1)  continue  ongoing  program-level  initiatives  through 
mid-November  2002  and  16  ongoing  projects  to  their  next  milestones 
and  (2)  start  6  new  projects.  Examples  of  new  projects  are 

•  Reporting  Compliance,  which  is  to,  among  other  things,  select 
returns  for  examination 

•  Filing  and  Payment  Compliance,  which  is  to  enable  access  to 
taxpayer  data  in  CADE  and  the  Custodial  Accounting  Project 
system  to  determine  compliance  and  allow  taxpayers  to  resolve 
account  issues  electronically 


•  Compared  to  the  total  BSM  funds  requested  in  the  third  and  fourth  plans 
combined,  the  fifth  plan  represents  an  18  percent  decrease  in  program 
management  spending  and  a  27  percent  increase  in  project  acquisition 
spending. 


•  The  third  and  fourth  plans  generally  provided  funding  for  FY  2001 ,  and 

the  fifth  plan  does  so  for  FY  2002. _ 
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i 


GAO 

Accountability  *  Integrity  *  Reliability 


Background 


3rd  and  4th  Plans  5th  Plan 

($343  million)  ($391  million) 


n  Program  Management  Spending  D  Project  Acquisition  Spending 
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GAO 

Accountability  *  Integrity  *  Reliability 


Background 


•  Like  its  previous  plans,  IRS’  fifth  expenditure  plan  covers  contractor 
costs,  such  as  the  Prime  Systems  Integration  Support  (PRIME) 
contractor  and  the  systems  engineering  and  technical  assistance 
contractor  (MITRE),  and  not  IRS  internal  costs,  such  as  IRS  BSM 
program  office  (BSMO)  staff  costs.  As  we  previously  reported,® 

•  IRS’  actual  use  of  prior  BSM  funding  has  been  limited  to  the 
modernization  program 

•  IRS’  actual  use  of  prior  IS  funding  has  included  modernization 
activities. 


^Internal  Revenue  Service:  Results  of  Review  of  IRS  Spending  for  Business  Systems  Modernization  (GAO-01-920,  August  17,  2001). 
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GAO 

Accountability  *  Integrity  *  Reliability 


Background 


Summary  of  Prior  GAO  Expenditure  Plan  Reviews 


•  To  date,  GAO  has  reviewed  and  reported  on  six  requests  for  BSM 
funding  releases.® 


•  Since  mid-1999,  we  have  reported^  on  the  risks  associated  with  IRS’ 
approach  of  concurrently  building  systems  while  developing  and 
implementing  program  management  capabilities  such  as  having  a 
fully  operational  program  management  office  and  implementing  the 
ELC. 


®For  details  on  our  past  review  results,  see  appendix  II. 

^For  example,  see  Business  Systems  Modernization:  Results  of  Review  of  IRS’  March  2001  Expenditure  Plan  (GAO-01-716,  June  29,  2001). 
and  Internal  Revenue  Service:  Progress  Continues  But  Serious  Management  Challenges  Remain  (GAO-01  -5627,  April  2,  2001 ). 
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GAO 

Accountability  *  Integrity  *  Reliability 


Background 


•  In  summary,  we  reported  that  attempting  to  acquire  modernized 
systems  before  having  the  requisite  management  capability 
increases  the  risk  that  systems  will  experience  cost,  schedule,  and 
performance  shortfalls. 


•  EA,  configuration  management,  quality  assurance,  and  risk 
management,  are  but  four  of  many  management  controls  required 
under  IRS’  ELC,  which  is  a  structured  method  for  managing  system 
modernization  program  and  project  investments  throughout  their  life 
cycles  (see  below  simplified  diagram  of  ELC). 
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i 


GAO 

Accountability  *  Integrity  *  Reliability 


Background 


IRS’  Enterprise  Life  Cycle  Phases  and  Milestones 


ELC  Phases 


X 


Enterprise 

Deployment 
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GAO 

Accountability  *  Integrity  *  Reliability 


Background 


•  We  have  also  reported®  that  the  risks  associated  with  building 
systems  without  the  requisite  management  controls  are  not  as 
severe  early  in  projects’  life  cycles  when  they  are  being  planned 
(project  definition  and  preliminary  system  design),  but  escalate  as 
projects  are  built  (detailed  design  and  development)  and 
implemented  (enterprise  deployment). 


•  In  the  case  of  IRS  and  its  ELC,  this  point  of  risk  escalation  is  ELC 
Milestone  3,  as  is  shown  in  the  following  graphic.  From  this  point 
through  deployment  (Milestone  4)  to  operations  and  support 
(Milestone  5),  risk  can  increase  significantly. 

•  In  June  2001  report,®  we  identified  key  IRS  projects  that  were 
approaching  or  had  passed  Milestone  3  that  were  beginning  to 
experience  such  cost,  schedule,  and  performance  shortfalls,  and 
concluded  that  program  risks  were  increasing  (see  graphic  on  next 
page). 


®For  example,  see  Tax  Systems  Modernization:  Results  of  Review  of  IRS’  Third  Expenditure  Plan  (GAO-01  -227,  January  22,  2001 ).  -10 

^Business  Systems  Modernization:  Results  of  Review  of  IRS’  March  2001  Expenditure  Plan  (GAO-01  -716,  June  29,  2001 ). 
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i 


GAO 

Accountability  *  Integrity  *  Reliability 


Background 


Timeline  Depicting  Escalating  Program  Execution  Risk  as  of  April  2001 


Program  Management  Capability 


-IRS  was  here 
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Results  in  Brief 

Accountability  *  Integrity  *  Reliability _ 

•  IRS’  fifth  plan  satisfies  each  of  six  legislative  conditions. 


Legislative  Conditions 

Satisfies 

Does  Not  Satisfy 

1 .  Meets  0MB  capital  planning  and  investment  control  review 
requirements. 

y 

2.  Complies  with  IRS’  enterprise  architecture. 

V' 

3.  Meets  the  requirements  of  IRS’  life  cycle  program. 

•/ 

4.  Approved  by  IRS,  Treasury,  and  0MB. 

5.  Reviewed  by  GAO. 

V' 

6.  Complies  with  federal  acquisition  requirements  and  management 
practices.'^ 

■/ 

•  While  the  plan  provides  for  satisfying  these  conditions,  IRS  still 
has  to  fully  implennent  the  controls  and  capabilities  described  in 
the  plan. 


^■•These  acquisition  requirements  and  practices  are  intended  to  establish  acquisition  management  rigor  and  discipline,  such  as  those  defined  in  the  20 
Software  Engineering  Institute’s  acquisition  model.  Our  analysis  of  the  plan  focused  on  satisfaction  of  this  model’s  tenets. 
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GAO 

Accountability  *  Integrity  *  Reliability 


Results  in  Brief 


•  Nevertheless,  IRS'  modernization  management  capability  is  still  not 
where  it  should  be  given  (1)  the  number  of  acquisition  projects  that  are 
underway,  (2)  the  fact  that  several  of  these  projects  have  already 
entered  the  critical  building  stage  of  their  life  cycles  (Milestone  3)  and 
are  to  begin  deployment  (Milestone  4)  during  calendar  year  2002,  and 
(3)  IRS’  plan  to  begin  additional  projects. 


•  Modernization  management  controls  and  capabilities  that  are  not  yet 
fully  implemented  are 

•  Software  acquisition  management 

•  Configuration  management 

•  Quality  assurance 

•  Risk  management 

•  EA  implementation 

•  Human  capital  management, 

•  Integrated  program  scheduling,  and 

•  Cost  and  schedule  estimating 
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GAO 

Accountability  *  Integrity  *  Reliability 


Results  in  Brief 


•  The  increased  risk  of  proceeding  without  these  controls  and  capabilities 
is  now  contributing  to  actual  project  cost,  schedule,  and  performance 
shortfalls.  For  example. 


•  In  the  fifth  plan,  IRS  reports  that  CADE’s  (Release  1)  Milestone  4 
date  has  slipped  by  6  months,  and  its  cost  has  increased  by  $5 
million  (13%  above  the  fourth  plan’s  funding  level),  and 


•  Since  submitting  the  fifth  plan  for  approval,  IRS  announced  that 
STIR’S  Milestone  4  date  has  slipped  by  3  months,  and  its  cost  has 
increased  by  $6  million  (24%  above  fourth  plan  funding  level). 
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GAO 

Accountability  *  Integrity  *  Reliability 


Results  in  Brief 


•  In  our  view,  reliance  on  a  combination  of  existing  immature  processes 
and  individual  expertise  and  heroic  efforts  is  a  short-term  solution  to  a 
long-term  need.  Given  that  these  controls’  immaturity  have  already 
contributed  to  project  cost,  schedule,  and  performance  shortfalls,  and 
will  likely  lead  to  future  shortfalls,  IRS  needs  a  better  strategy  for 
mitigating  the  risks  it  faces  in  implementing  its  fifth  expenditure  plan. 


•  To  assist  IRS  in  striking  a  proper  balance  between  the  need  to  quickly 
introduce  modernized  systems  and  prudently  manage  the  risks  inherent 
in  doing  so,  we  are  making  recommendations  to  the  Commissioner  of 
Internal  Revenue. 
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GAO 

Accountability  *  Integrity  *  Reliability 


Results 


Objective  1 :  Fifth  plan  satisfies  the  conditions  in  IRS’  FY  2002 
appropriations  act. 


Legislative  Conditions 

Expenditure  Plan  Provisions 

1.  Meets  the  0MB  capital 
planning  and  IT 
investment  control 
review  requirements. 

IRS’  fifth  expenditure  plan  provides  for  managing 
investments  as  part  of  a  portfoMo  through  its 
Investment  Decision  Management  process.  This 
includes  conducting  periodic  portfolio  reviews  to 
assess  changes  in  business  priorities  and  project 
schedules.  The  fifth  plan  provides  for  such  an 
assessment  in  January  2002.  The  fifth  plan  also 
provides  for  IRS  to  revise  and  implement  its 
business  case  guidance  to  achieve  better  system 
investment  decisions  based  on  compelling  retum- 
on-investment  justifications. 
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i 


GAO 

Accountability  *  Integrity  *  Reliability 


Legislative  Conditions 

Expenditure  Plan  Provisions 

2.  Complies  with  IRS’ 
enterprise 
architecture. 

The  fifth  plan  provides  funds  to  continue  definition 

and  implementation  of  the  enterprise  architecture. 

For  example,  it  provides  for 

•  addressing  issues  raised  during  review  and 
approval  of  EA  releases  1.0  and  1.1 

•  completing  and  issuing  EA  release  2.0 

•  addressing  EA  compliance  certification  for 
selected  projects 

•  issuance  of  the  2002  and  2003  release 
architectures 

•  operation  of  the  architecture  engineering  office 

3.  Meets  the 

requirements  of  IRS’ 
life  cycle  program. 

The  plan  provides  funds  for  meeting  the 
requirements  in  IRS’  life  cycle  management 
program,  which  IRS  refers  to  as  ELC.  For  example, 
the  plan  calls  for 

•  maintaining  responsibility  for  coordinating, 
tracking,  and  integrating  all  program-wide  costs, 
schedules,  releases,  issues,  and  risks 

•  maintaining  the  ELC 

•  completing  the  implementation  of  configuration 
management,  including  establishing 
configuration  items  for  existing  (legacy) 
systems  impacted  by  near-term  modernization 
projects  (project  releases  to  be  deployed  in  2002 
and  2003),  and  developing  and  controlling  a 
configuration  management  master  log. 

Results 
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i 


GAO 

Accountability  *  Integrity  *  Reliability 


Results 


Legislative  Conditions 

Expenditure  Plan  Provisions 

4.  Approved  by  IRS, 
Treasury,  and  0MB. 

•  IRS  -  September  17,  2001 

•  Treasury  -  October  19,  2001 

•  OMB  -  October  29,  2001 

•  Submitted  to  IRS’  appropriations 
subcommittees  -  November  13,  2001 

6.  Reviewed  by  GAO. 

•  GAO  -  December  10,  2001  briefing  to  IRS’ 

House  appropriation  subcommittee 
-  December  11,  2001  briefing  to  IRS’ 

Senate  appropriation  subcommittee 

6.  Complies  with  the 
acquisition  rules, 
requirements, 
guidehnes,  and 
systems  acquisition 
management  practices 
of  the  federal 
Government. 

As  part  of  the  ELC,  IRS  has  defined  processes, 
roles,  responsibihties,  etc.  for  implementing 

Software  Engineering  Institute  (SEI)  Software 
Acquisition  Capability  Maturity  Model™  practices 
within  the  level  2  key  process  areas.'^  These 
practices  are  consistent  with  federal  acquisition 
requirements  and  management  practices,  and  the 
plan  calls  for  implementation  of  the  ELC  on  all 
projects. 

■•^These  are  Acquisition  Planning,  Solicitation,  Requirements  Development  and  Management,  Project  Management,  Contract  Tracking 
and  Oversight,  Evaluation,  and  Transition  to  Support. 
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GAO 

Accountability  *  Integrity  *  Reliability 


Results 


Objective  2:  Despite  important  progress,  key  controls 
and  capabilities  have  not  yet  been 
implemented. 

•  Since  we  reported  on  IRS’  last  plan,''^  IRS  has  made  important  progress 
in  implementing  modernization  controls  and  capabilities  and  addressing 
our  recommendations. 

•  However,  key  controls  and  capabilities  are  still  not  fully  implemented. 


IRS  Commitments  Made  in  March  2001  and  Previous 
Plans  to  Address  Weaknesses  and  Recommendations 

Completed 

Not  Completed 

Revised  Commitments  in  November 
2001  Plan 

Mature  software  acquisition  capabilities 

■  Have  all  projects  follow  SA  CMM  level  2  processes  by 
September  2001. 

(See  pp.  30-32) 

All  projects  to  satisfy  SA  CMM  level  2 
requirements  during  FY  2004. 

■  Perform  internal  SA  CMM  level  2  capability 
compliance  assessment  by  September  2001. 

(See  pp.  30-32) 

Internal  assessment  by  June  2002. 

■  Have  an  external  evaluation  (SCE)  performed  by 
independent  assessor  in  March  2002  to  ensure 
compliance  with  SEI’s  SA  CMM  level  2  requirements. 

(Seep.  30-32) 

External  evaluation  in  December  2002. 

'^^Business  Systems  Modernization:  Results  of  Review  of  IRS’  March  2001  Expenditure  Plan  {GAO-01  -71 6,  June  29,  2001 ). 
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1  A  Rssults 

^  G  A  O 

Accountability  *  Integrity  *  Reliability 


IRS  Commitments  Made  in  March  2001  and  Previous 

Completed 

Not  Completed 

Revised  Commitments  in  Nov.  2001 

Plans  to  Address  Weaknesses  and  Recommendations 

ELC  definition  and  implementation  (Configuration 

Plan 

■  Fully  define  and  institutionalize  PRIME  standard 

■  Fully  define  and  institutionalize  BSMO  standard 
configuration  management  nrocedures  bv  Julv  200 1 . 

To  be  completed  by  early  2002. 

■  Establish  configuration  item  baselines  for  2002  Release 

■  Identify  configuration  items  for  current  production 
environment  impacted  by  near-term  (2002  and  2003) 

(See  pp.  35-36) 

2002  project  releases  by  March  2002. 

2003  project  releases  date  not  yet 

■  Establish  a  centralized  CM  repository  by  November 

2001. 

rSeepp  35-36) 

To  be  completed  by  early  2002. 

Oualitv  Assurance: 

■  Implement  BSM  program  office  quality  assurance 
function  by  November  2001. 

Risk  Management: 

■  Complete  definition  of  risk  management  program  by 
November  2001. 

■  Complete  initial  training  of  BSMO/PRIME  personnel  by 
November  2001. 

■  Fully  implement  risk  management  program  by 

November  2001. 

(See  pp.  41-42) 

To  be  completed  by  February  2002. 

Enterprise  Architecture  (EA) 

■  Approve  EA  1.1. 

■  Ensure  ongoing  projects  are  aligned  with  EA  in 
accordance  with  compliance  certification  process. 

(See  p.  44-45) 

To  be  completed  by  late-December 

2001. 
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Accountability  *  Integrity  *  Reliability 


IRS  Has  Not  Implemented  Effective  Software  Acquisition 
Processes 


Results 


•  The  Clinger-Cohen  Act  requires  the  establishment  of  effective  IT 
management  processes.  SEI’s  Software  Acquisition  Capability 
Maturity  Model™  defines  such  processes  for  managing  software 
acquisitions.  Since  1995,  we  have  recommended  that  IRS 
establish,  at  a  minimum,  the  “repeatable”  level  of  SEI’s  software 
acquisition  management  processes  (Level  2).''4 


•  IRS  has  not  completed  implementation  of  SEI’s  Level  2  software 
processes,  although  it  committed  in  its  fourth  plan  to  do  so  by 
September  2001. 


^‘^Tax  Systems  Modernization:  Management  and  Technical  Weaknesses  Must  Be  Corrected  If  Modernization  Is  to  Succeed  30 

(GAO/AIMD-95-156,  July  26,  1995). 
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GAO 
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Results 


•  IRS  officials  attribute  the  delay  to  their  underestimation  of  how  long 
it  takes  to  implement  the  practices. 


•  To  effectively  implement  mature  acquisition  management 
processes,  an  organization  needs  to 

•  first  assess  its  strengths  and  weaknesses  and 

•  then  develop  a  plan  for  leveraging  existing  strengths  and 
correcting  weaknesses,  paying  special  attention  to  projects  that 
are  most  critical  and  vulnerable  to  weaknesses. 


•  To  address  its  acquisition  management  control  weaknesses,  IRS 
has  recently 

•  engaged  SEI  to  help  better  estimate  how  long  this  will  take,  and 

•  developed  a  plan  for  implementing  these  practices  which 
includes 
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Results 


•  focusing  first  on  two  new  BSM  projects, 

•  having  these  two  projects  internally  assessed  by  June  2002, 

•  having  these  two  projects  independently  evaluated  by 
December  2002,  and 

•  implementing  the  practices  on  remaining  BSM  projects  during 
FY  2004. 


•  In  effect,  this  plan  means  that  IRS  will  not  implement  mature 
acquisition  processes  on  key  projects  until  after  initial  system 
increments  have  passed  critical,  later  life  cycle  phases.  As  a  result, 
IRS  will  continue  to  run  the  risk  that  these  projects’  promised 
capabilities  will  not  be  delivered  on  time  and  within  budget. 


•  The  significance  of  this  risk  is  magnified  in  light  of  the  multiple 
dependencies  that  exist  among  projects,  where  for  example,  delays 
in  one  can  cause  cascading  delays  in  others. 
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Results 


IRS  Has  Made  Important  Progress,  But  Does  Not  Yet  Have  Effective 
Configuration  Management 


•  Effective  configuration  management  is  an  essential  control  for 
ensuring  the  integrity  and  consistency  of  system  modernization 
program  and  project  products  throughout  their  life  cycles. 


•  In  June  2001 ,  we  reported^®  BSM  configuration  management  was 
ineffective.  Accordingly,  we  made  recommendations  to  address  this 
weakness. 


^^Business  Systems  Modernization:  Results  of  Review  of  IRS’  March  2001  Expenditure  Plan  {GAO-01-716,  June  29,  2001). 
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Accountability  *  Integrity  *  Reliability 


•  IRS  has  made  important  progress  in  addressing  our 
recommendations.  For  example,  it  has 


Results 


•  identified  configuration  items  and  components  to  be  controlled  for 
BSM  projects, 

•  had  the  PRIME  revise  its  program  configuration  management 
plan  and  procedures,  which  was  completed  in  July  2001 , 

•  trained  contractor  and  selected  BSM  program  office  personnel  in 
configuration  management, 

•  established  baselines  for  approved  program  and  project 
products, 

•  initiated  periodic  internal  configuration  management  process 
audits  on  selected  projects  to  ensure  plans  and  procedures  are 
being  followed,  and 

•  established  configuration  control  boards,  including  defining 
thresholds  governing  which  board  should  address  which  request. 


34 


Page  39 


GAO-02-356  IRS's  November  2001  Expenditure  Plan 


Appendix  I 

Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 
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Results 


•  However,  effective  configuration  management,  as  we  previously 
recommended,  requires  an  enterprise-wide  configuration 
management  process,  including  plans  and  procedures  governing 
PRIME,  BSM  program  office,  and  IRS  IT  services  organization 
products  and  systems.  At  a  minimum,  this  also  involves 


•  establishing  an  integrated  program-wide  configuration  library  (i.e., 
a  repository). 


•  defining  and  implementing  a  program-wide  configuration  status 
account  reporting  mechanism,  and 


•  identifying  configuration  items  and  components  for  current 
(legacy)  systems  affected  by  near-term  (2002  and  2003)  BSM 
project  releases. 
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Results 


•  IRS  officials  acknowledge  these  steps  have  not  been  completed  but 
have  plans  to  do  so  by  March  2002,  except  for  the  2003  project 
releases,  for  which  a  date  has  not  yet  been  established. 


•  IRS  officials  also  stated  that  these  steps  have  not  yet  been  fully 
implemented  because  all  the  steps  needed  to  implement  an  effective 
configuration  management  process  were  not  originally  understood. 


•  While  IRS  has  made  progress  in  implementing  effective 
configuration  management,  until  it  fully  does  so,  IRS  cannot 
adequately  assure  that  systems  are  being  designed  and  developed 
in  accordance  with  enterprise-wide  needs  and  requirements,  and 
thus  the  likelihood  that  projects  will  eventually  require  expensive  re¬ 
work  is  increased. 
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Accountability  *  Integrity  *  Reliability 


Results 


IRS  Quality  Assurance  Efforts  Have  Identified  PRIME  Problems 


•  Quality  assurance  (QA)  provides  independent  assessments  of  whether 
management  process  requirements  are  being  followed  and  whether 
product  standards  and  requirements  are  being  satisfied. 


•  In  2000,  IRS  established  a  QA  organization  within  its  BSM  program 
office  to,  among  other  things,  determine  how  well 

•  the  BSM  program  office  was  following  ELC  acquisition  management 
processes  and  product  standards  and 

•  the  PRIME’S  QA  function  was  performing  its  role  of  assessing  the 
PRIME’S  adherence  to  quality  processes  and  standards  in 
producing  BSM  deliverables. 
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Results 


•  IRS  reviewed  the  PRIME’S  QA  function,  and  in  March  2001 ,  reported  5 
major  findings  and  concluded  that  the  PRIME’S  QA  was  not  always 
effective.  Examples  of  findings  were  that  the  PRIME’S  QA 


•  had  not  planned  its  work  adequately,  which  contributed  to 
inadequate  process  audits  and  product  reviews, 

•  had  not  defined  program  and  project  metrics  for  assessing 
process  and  product  quality,  and 

•  was  not  organizationally  independent. 


•  To  address  these  weaknesses,  IRS  defined  24  corrective  actions  for  the 
PRIME.  As  of  November  2001 ,  IRS  reported  that  the  PRIME  had 
implemented  7  of  the  corrective  actions,  but  17  remained  open. 
Examples  of  open  actions  are 

•  Ensure  that  QA  reviews  and  approves  deliverables  before  they 
are  transmitted  to  IRS 
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Results 


•  Require  PRIME  program  and  project  quality  plans  to  adhere  to  ELC 
standards. 


•  Until  the  PRIME  corrects  these  weaknesses,  the  probability  of  PRIME 
deliverables  not  meeting  expectations  is  unnecessarily  increased,  and 
the  level  of  IRS  contract  oversight  and  control  needed  to  compensate  for 
these  weaknesses  increases. 
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Results 


Despite  Progress.  Risk  Management  Not  Yet  Fully  Implemented 


•  A  risk  represents  a  potential  problem.  The  purpose  of  risk  management 
is  to  identify  program  and  project  risks  before  they  result  in  a  problem, 
appropriately  respond  to  a  risk  based  on  its  significance  (probability  of 
occurrence  times  its  potential  impact),  and  to  actively  manage  it. 


•  Effectively  managing  these  risks  is  one  way  to  minimize  the  chances  of 
program  and  project  cost,  schedule,  and  performance  problems 
occurring.  Consistent  with  our  efforts  to  constructively  engage  with  IRS 
on  BSM,  in  1999  we  orally  recommended  to  IRS’  former  CIO  that  IRS 
implement  this  capability  across  the  BSM  program  and  projects. 
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Results 


•  IRS  has  defined  risk  management  policies  and  procedures  for  its  ELC. 
These  included  processes  to 

•  identify  risks, 

•  evaluate  probability  of  occurrence  and  potential  impact  on  each 
project  and  across  the  BSM  program, 

•  develop  risk  mitigation  plans  and  track  their  disposition, 

•  establish  a  repository  to  maintain  an  inventory  of  identified  risks,  and 

•  report  on  the  status  of  risks  and  mitigation  plans. 


•  Despite  these  steps,  IRS  has  yet  to  fully  implement  its  risk  management 
program  policies  and  procedures.  For  example, 

•  IRS’  inventory  of  BSM  risks  does  not  identify  all  known  risks  (e.g. 
lack  of  mature  software  processes,  lack  of  effective  configuration 
management). 
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Results 


•  Risk  plans  and  their  disposition  are  not  in  all  cases  being  tracked 
and  updated. 


•  To  strengthen  risk  management,  IRS  hired  a  senior  executive  in  June 
2001  to  implement  effective  risk  management.  This  official 

•  attributed  IRS’  delay  in  implementing  risk  management  to  a  lack  of 
incentives  to  report  risks,  and 

•  created  an  action  team  to  review  IRS’  risk  management  procedures 
and  develop  a  plan  for  “revitalizing”  the  program,  including 
incorporating  incentives  for  escalating  risks. 


•  IRS  plans  to  have  risk  management  fully  implemented  by  February 
2002. 


•  Until  this  is  done,  IRS  cannot  assure  it  is  adequately  managing  program 
and  project  risks.  Thus,  the  likelihood  of  BSM  projects  experiencing 
further  cost,  schedule,  and  performance  shortfalls  is  increased. 


42 


Page  47 


GAO-02-356  IRS's  November  2001  Expenditure  Plan 


Appendix  I 

Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 


GAO 

Accountability  *  Integrity  *  Reliability 


Results 


IRS  Has  Established  Processes  To  Certify  Alignment  of  BSM  Projects 
With  EA,  But  Key  Projects  Yet  To  Be  Certified 


•  EA  is  an  institutional  blueprint  defining  how  an  enterprise  operates 
today,  in  both  business  and  technology  terms,  and  how  it  wants  to 
operate  at  some  point  in  the  future.  An  EA  also  includes  a  roadmap  for 
transitioning  between  these  environments. 

•  Since  1995''®,  we  have  recommended  that  IRS 

•  define  and  implement  an  EA  to  guide  and  constrain  the  acquisition 
of  its  modernized  systems  and 

•  map  BSM  projects  to  the  EA  to  ensure  they  are  built  in  accordance 
with  the  EA,  which  reduces  the  risk  of  expensive  rework,  especially 
after  projects  have  begun  detailed  design  and  development. 

•  We  reported  in  our  June  2001  report^^  that  while  IRS  has  developed  EA 
versions  1 .0  and  1.1,  it  had  not  yet  performed  the  requisite  mappings. 


^®For  example,  Tax  Systems  Modernization:  Management  and  Technical  Weaknesses  Must  Be  Corrected  If  Modernization  Is  to  Succeed 
(GAO/AIMD-95-156,  July  26,  1995). 

Business  Systems  Modernization:  Results  of  Review  of  IRS’  March  2001  Expenditure  Plan  {GAO-01-716,  June  29,  2001). 
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Results 


•  Accordingly,  we  also  recommended  that  IRS  not  approve  projects 
exiting  Milestone  3  until  the  required  assessments  were  performed 
to  certify  that  projects  are  aligned  with  the  EA. 


•  IRS  has  taken  important  steps  to  implement  this  recommendation. 
For  example 

•  In  September  2001 ,  IRS  issued  processes  for  certifying  project 
compliance  with  the  EA  at  Milestone  3,  before  projects  proceed 
with  detailed  design  and  development. 


•  Nevertheless,  IRS  has  not  yet  certified  all  BSM  projects  that  are  at 
or  past  Milestone  3  via  this  process  and  the  current  EA  version, 
including 

•  Security  and  Technology  Infrastructure  Release, 

•  Internet  Refund  and  Fact  of  Filing  (IRFoF),  and 

•  Custodial  Accounting  Project. 
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Results 


•  IRS  officials  stated  these  projects  were  delayed  in  being  certified 
because  issuing  a  draft  of  EA  version  2.0  in  October  2001  for 
comment  was  their  highest  priority.  Also,  STIR  and  CAP  were 
certified  compliant  with  an  earlier  EA  version.  IRS  plans  to  complete 
certification  of  these  projects  by  late-December  2001 . 


•  However,  until  this  is  done,  IRS  cannot  provide  assurance  that  the 
project  requirements  and  design  are  properly  aligned  with  the  EA. 
Without  this  alignment  being  established  early  and  continuously 
throughout  a  project’s  life  cycle,  IRS  increases  the  chances  that 
expensive  project  rework  will  be  required  to  meet  IRS  business  and 
systems  needs. 
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Results 


Weaknesses  in  Controls  and  Capabilities  Increase  Risks 

•  Weaknesses  in  any  one  of  the  aforementioned  modernization 
management  controls  introduces  an  unnecessary  element  of  risk  to 
the  BSM  program,  but  the  combination  of  these  weaknesses 
introduces  a  level  of  risk  that  increases  exponentially  over  time.  Given 
that  the  fifth  plan  provides  funding  for  later  phases  of  key  projects, 
continued  development  of  other  projects,  and  starting  of  new  projects, 
it  is  likely  that  BSM  projects  will  encounter  additional  cost,  schedule, 
and  performance  shortfalls.  This  combination  of  circumstances  and 
events  is  represented  in  the  following  updated  graphic. 

•  As  discussed  later  in  this  briefing,  IRS  reported  in  the  fifth  plan  that 
BSM  projects  have  already  encountered  cost,  schedule,  and/or 
performance  shortfalls  in  meeting  commitments  made  in  the  fourth 
plan.  In  those  cases  where  the  fifth  plan  cited  a  reason  for  the 
shortfall,  our  analysis  of  the  reasons  showed  that  weak  management 
controls  were  a  contributor-either  via  (1)  proactive  prudent  IRS 
decision  making  not  to  start  or  continue  projects  because  of  immature 
controls  (for  example,  see  Reporting  Compliance  on  p.  51  of  this 
briefing)  or  (2)  immature  controls  allowed  project  shortfalls  to  occur. 
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Results 


Current  Timeline  Deoictina  Escalatina  Proc 

as  of  December  2001 

IRS  is  here - 

Program  Management  Capability  ' 

Program  Management  Office 

iram  Execution  Risk 

Selected  Management  Capabilities  (e.g.  Configuration  Mgmt,  Quality  Assurance,  Risk  Mgmt) 

/X  I 

Enterprise  Architecture  1.0  1.1  J 

Selected  Key  Projects'* 

./  . 

/  t  J 

^  e-Services  \  ^ 

\  ^ 

Other  projects  (18)  1 

1  1  1  1  1  1  \/1  1  1  1  1  1  1  1  1  1  1  1 
1/00  1/01 

1  1  1  1  1  1  1  1  1  1 

1/02  10/02 

Denotes  Milestone  3  -  beginning  of  detailed  design  and  development  ^  Denotes  management  capability  fully  established 

4  of  22  total  acquisition  projects  Denotes  Milestone  4  -  completion  of  detailed  design  and  development  /  beginning  of  deployment 
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Results 


•  IRS  acknowledges  these  risks.  According  to  the  CIO,  until  the 
weaknesses  are  fully  addressed,  IRS  is  mitigating  them  by 

•  relying  on  existing  immature  processes, 

•  leveraging  the  knowledge,  skills,  and  abilities  of  experienced 
senior  executives  to  make  sure  project  issues  are  proactively 
managed, and 

•  hiring  additional  experienced  executives. 


•  In  our  view,  reliance  on  a  combination  of  immature  processes  and 
individual  capabilities  and  heroic  efforts  is  not  a  recipe  for  success. 
Past  government  and  industry  experience  shows  the  probability  of 
repeating  successes  on  projects  using  this  approach  is  low. 
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Results 


Objective  3:  Other  observations  about  IRS’  fifth  plan 
and  its  BSM  program 


Observation  1 :  Plan  Discloses  and  Explains  Project  Cost 
and  Schedule  Changes.  But  Continues  to  Omit  Changes  to 
System  Capabilities  and  Expected  Benefits 

•  In  our  June  1999  report  on  IRS’  first  plan,^^  we  recommended  that 
IRS,  in  future  expenditure  plans,  report  progress  against  incremental 
project  commitments.  Since  then,  we  have  reported  with  each  plan 
that  IRS  has  improved  its  reporting  but  has  not  included  progress  on 
all  incremental  project  commitments,  such  as  promised  system 
capabilities  and  expected  system  benefits. 


•  In  the  fifth  plan,  IRS  disclosed  that  18  projects  have  experienced  cost 
and/or  schedule  shortfalls  against  commitments  made  in  its  fourth  and 
other  prior  plans. 


^^Tax  Systems  Modernization:  Results  of  Review  of  IRS’  Initial  Expenditure  Plan  (GAO/AMD-99-206,  June  15,  1999).  49 
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•  Of  the  18, 


Results 


•  7  were  program  management  initiatives  that  had  cost 
increases  ranging  from  1%  to  84%  ($150,000  to  $1.9  million), 
and 


•  1 1  were  system  acquisition  projects,  of  which 

•  10  experienced  schedule  delays  ranging  from  1.5  months 
to  14  months;  6  of  the  10  experienced  delays  of  4  months 
or  more;  and  1  project  had  its  schedule  shortened  by  4 
months. 

•  8  experienced  cost  increases  ranging  from  4%  to  66% 
($500,000  to  $6.1  million);  3  had  cost  decreases  ranging 
from  4%  to  23%  ($365,000  to  $4.1  million). 


50 


Page  55 


GAO-02-356  IRS's  November  2001  Expenditure  Plan 


Appendix  I 

Briefing  Slides  from  December  10  and  11, 
2001,  Briefings  to  the  Senate  and  House 
Appropriations  Subcommittee  Staffs 


i 


GAO 

Accountability  *  Integrity  *  Reliability 


•  Examples  include^o 


Results 


Program/  Project 
Management  Initiative 

Commitment  Date  and 
Funding  as  of  3/2001 
($000) 

Revised  Commitment 
Date  and  Funding 
($000) 

Change  (%) 

Reporting  Compliance- 

11/30/01 

11/30/02 

+  12  months 

Individual 
Milestone  2 

$6,022 

$10,000 

+3,978  (66%) 

Customer  Account  Data 

12/31/01 

6/30/02 

+6  months 

Engine  (CADE) 
Milestone  4 

$40,038 

$45,338 

+$5,300  (13%) 

Security  and 

9/30/01 

1/31/02 

+4  months 

Technology 
Infrastructure  Releases 
(STIR) 

Milestone  4 

$25,178 

$31,287 

+$6,109  (24%) 

•  However,  the  fifth  plan  does  not  provide  the  level  of  specificity 
needed  to  identify  changes,  if  any,  to  projects’  scope.  For 
example,  while  the  plan  discusses  the  impact  of  the  schedule 
and  cost  variances  on  each  project,  it  does  not  specify 

(1)  changes  to  project  scope  and  related  benefit  expectations  or 

(2)  effects  on  interdependent  projects  and  their  benefits. 


list  of  18  is  in  appendix  III. 
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Results 


•  In  addition,  since  the  plan  was  completed,  other  slippages  have 
occurred.  For  example,  on  October  23,  2001 ,  IRS  moved  STIR’s 
Milestone  4  from  October  2001  to  January  2002,  increasing  its 
reported  schedule  slippage  from  1  months  to  4  months. 


•  According  to  IRS  officials,  the  omission  of  such  scope  and  benefit 
information  was  an  oversight,  and  they  plan  to  correct  it  in  the 
next  expenditure  plan. 


•  By  not  fully  providing  scope  and  benefit  change  information, 
congressional  oversight  of  IRS  modernization  management 
performance  and  accountability  is  constrained. 
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Results 


Observation  2:  IRS  Beginning  to  Develop  BSM  Human 
Capital  Strategy 


•  As  we  have  previously  reported^^  strategic  human  capital 
centers  on  viewing  people  as  assets  whose  value  to  an 
organization  can  be  enhanced  through  investment.  As  the  value 
of  people  increases,  so  does  the  performance  capacity  of  the 
organization.  To  maintain  and  enhance  the  capabilities  of  IT 
staff,  organizations  should,  among  other  things. 


•  assess  knowledge  and  skills  needed  to  effectively  perform  IT 
operations  to  support  agency  mission  and  goals, 

•  inventory  the  knowledge  and  skills  of  current  IT  staff, 

•  identify  gaps  between  requirements  and  current  staffing,  and 

•  develop  and  implement  plans  to  fill  the  gaps. 


Human  Capital:  Attracting  and  Retaining  a  High-Quaiity  information  Technology  Workforce  53 

CGAO-02-1 13T,  October  4,  2001 ) 
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Results 


•  IRS  has  begun  to  address  this  issue.  For  example,  it  has  hired  a 
human  capital  specialist  to  develop  a  plan  by  January  2002,  for 
defining  and  implementing  an  IT  human  capital  strategy. 


•  Until  IRS  develops  and  implements  this  strategy,  it  will  not  know 
whether  it  has  the  right  IT  knowledge  and  skills  to  effectively 
manage  the  BSM  program.  Without  this,  IRS  increases  the  risk  of 
BSM  program  and  project  cost,  schedule,  and  performance 
shortfalls. 
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Observation  3:  IRS’  Integrated  Master  Schedule  Has  Yet 
to  be  Finalized 


•  The  Integrated  Master  Schedule  is  an  important  tool  for  managing 
IRS’  acquisition  of  modernized  systems.  According  to  IRS,  it  is  to 

•  specify  about  20,000  major  tasks  and  associated  schedules, 
involving  a  dozen  organizations,  for  acquiring  and  implementing 
the  portfolio  of  BSM  projects,  and 

•  identify  the  numerous  and  complex  dependencies  across  these 
projects. 


•  IRS  officials  recognize  the  importance  of  having  such  an  integrated 
schedule  and  committed  to  completing  one  before  they  begin 
implementing  FY  2002  system  releases  (scheduled  to  b^in  in 
January  2002).  However,  despite  important  progress,  IRS  has  not 
yet  completed  the  schedule  because 

•  implementation  plans  (through  milestone  5)  have  not  yet  been 
developed  for  all  BSM  projects  to  be  implemented  in  FY  2002, 
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Results 


•  all  change  requests  that  impact  BSM  projects  baselines  have  not 
been  analyzed,  approved,  and  incorporated  into  the  schedule, 
and 

•  33  of  61  changes  that  need  to  be  made  to  existing  (legacy) 
systems  have  not  yet  been  analyzed,  approved,  and  incorporated 
into  the  schedule. 


•  IRS  officials  stated  that  they  plan  to  have  the  Integrated  Master 
Schedule  completed  by  late  December  2001 . 


•  They  attributed  the  delay  in  finalizing  the  schedule  to  this  being  the 
first  time  IRS  has  attempted  to  develop  a  management  tool  of  this 
size  and  complexity,  and  it  has  taken  longer  than  anticipated. 


•  Until  these  steps  are  completed,  IRS  does  not  have  the  means  to 
adequately  manage  the  interdependencies  within  and  among 
projects,  thus  increasing  the  risk  of  project  delays,  overruns,  and 
rework. 
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Results 


Observation  4:  IRS  Has  Not  Yet  Implemented  Effective 
Project  Cost  and  Schedule  Estimating  Practices 


•  Producing  reliable  estimates  of  expected  costs  for  program-level 
activities  and  projects  is  essential  to  determining  a  project’s  cost- 
effectiveness.  Without  this  information,  the  likelihood  of  poor 
investment  decisions  is  increased. 


•  As  part  of  our  ongoing  constructive  engagement  with  IRS,  we  have 
orally  recommended  to  the  former  and  current  IRS  CIOs  that  IRS 
adopt  effective  cost  and  schedule  estimating  practices.  SEI  defines 
such  practices  in  its  model  for  evaluating  organizational  cost  and 
schedule  estimating  capabilities. 22  The  practices  include  having 

•  a  historical  database, 

•  structured  processes  for  estimating  product  size  and  reuse, 

•  extrapolation  mechanisms. 


22Software  Engineering  Institute  Checklists  and  Criteria  for  Evaluating  the  Cost  and  Scheduie  Estimating  Capabilities  of  Software  57 

Organizations  (CMU/SEI-95-SR-005). 
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Results 


•  audit  trails, 

•  integrity  in  dealing  with  dictated  costs  and  schedules,  and 

•  data  collection  and  feedback  processes. 


•  IRS  officials  recognize  the  importance  of  each  of  these  practices. 
However,  IRS  is  not  currently  performing  them. 


•  Consequently  and  as  has  been  the  case  in  the  prior  plans,  the  cost 
and  schedule  estimates  in  IRS’  fifth  plan  are  contractor-provided, 
“rough  order  of  magnitude”  estimates,  that  have  not  been  subjected 
to  meaningful,  reliable  validation  by  IRS. 


•  IRS  officials  stated  that  heretofore  they  have  not  been  able  to 
dedicate  time  to  implementing  effective  estimating  practices 
because  of  other  competing  program  and  project  priorities. 
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Results 


•  However,  IRS  has  tasked  the  PRIME  to  develop  a  plan  to  implement  an 
estimating  capability,  that  is  to  include 

•  Selection  of  an  estimating  method, 

•  Development  of  a  plan  for  implementing  the  method  by  mid-January 
2002,  and 

•  Implementation  of  the  plan  beginning  by  the  end  of  February  2002. 


•  IRS  then  plans  to  develop  and  implement  an  approach  to  oversee 
PRIME  estimating  efforts  by  February  2002. 


•  IRS  officials  acknowledge  that  IRS’  lack  of  an  effective  estimating 
process  has  contributed  to  project  delays  and  cost  overruns.  Without 
improved  estimation  practices,  these  project  shortfalls  are  likely  to 
continue. 
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Accountability  *  Integrity  *  Reliability _ 

•  IRS’  fifth  plan  also  does  not  fully  provide  whether  projects’  scope  and 
expected  benefit  commitments  have  changed.  Such  information  is 
critical  to  congressional  oversight  of  IRS  modernization  management 
performance  and  accountability. 
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Recommendations  for 
Executive  Action 


To  address  the  escalating  risks  facing  IRS  on  its  BSM  program,  we 
recommend  that  the  Commissioner  of  Internal  Revenue 


•  reconsider  the  planned  scope  and  pace  of  the  BSM  program  as 
defined  in  the  fifth  expenditure  plan  with  the  goal  of  better  balancing 
the  number  of  system  acquisition  projects  underway  and  planned 
with  IRS'  capacity  to  manage  this  workload.  At  a  minimum,  the 
Commissioner’s  reconsideration  should  include 

•  slowing  ongoing  projects  and/or  delaying  new  project  starts  to 
reduce  BSMO  resource  demands, 

•  making  correcting  modernization  management  weaknesses  a  top 
priority  and  a  matter  of  top  management  attention,  and 

•  reapplying  resources  (financial  and  human  capital)  available  from 
slowed  and  delayed  projects  toward  correction  of  control 
weaknesses. 
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Recommendations  for 
Executive  Action 


•  To  that  end,  we  further  recommend  that  the  Commissioner  do  the 
following  with  respect  to  each  of  modernization  management 
weaknesses  that  we  identified. 


•  First,  for  software  acquisition  management, 

•  immediately  assess  CADE,  STIR,  and  e-Services  against  SEI 
SA-CMM  level  2  requirements, 

•  based  on  this  assessment,  develop  a  plan  for  correcting  identified 
weaknesses  for  these  projects,  including  having  an  independent 
CMM  evaluation  performed  on  these  projects  before  submission 
of  the  next  BSM  expenditure  plan, 

•  submit  with  the  next  expenditure  plan,  the  results  of  this 
independent  evaluation,  along  with  a  plan  for  ensuring  that  all 
BSM  projects  that  have  passed  milestone  3  will  meet  CMM  level 
2  requirements,  and 

•  require  all  projects  that  have  not  passed  milestone  3  as  of 
December  31 , 2001 ,  to  be  assessed  as  CMM  level  2  and  have  a 
plan  for  correcting  any  project  weaknesses  found  as  a  condition 
of  milestone  3  approval. 
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Recommendations  for 
Executive  Action 


•  Second,  for  configuration  management,  risk  management,  EA 
implementation,  human  capital  strategic  management,  integrated 
program  scheduling,  and  cost  and  schedule  estimating,  ensure  that 
commitments  discussed  in  this  briefing  for  addressing  residual 
weaknesses  are  implemented  as  planned,  and  report  any 
deviations,  from  these  planned  commitments  to  IRS’  appropriations 
subcommittees. 


•  Third,  until  PRIME  quality  assurance  weaknesses  are  corrected, 
increase  the  level  of  IRS  oversight,  scrutiny,  and  quality  assurance 
of  PRIME  activities. 
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Recommendations  for 
Executive  Action 


•  In  addition,  to  allow  for  effective  congressional  oversight  of  the  program, 
we  reiterate  our  prior  recommendation  that  the  Commissioner  report  to 
IRS’  appropriations  subcommittees  on  any  changes  to  expenditure  plan 
commitments  concerning  system  requirements/capabilities  to  be 
delivered  and  the  associated  benefits  to  be  realized,  and  continue  to 
report  such  performance  measures  in  future  expenditure  plans. 
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Appendix  I: 
IRS’  Expenditure  Plan 


BSM  Spending  Plan,  Fiscal  Year  2002  ($000) 


Proposed  Modernization  Initiatives 

Milestone 

Milestone 

Amount 

Date 

Requested 

Program  Level  Activities* 

Program  Management 

FY 

Nov.  02 

$7,918 

(formerly  Prime  Program  Management  Office) 

Business  Integration 

FY 

Nov.  02 

$10,461 

Architecture  &  Integration 

FY 

Nov.  02 

$32,539 

Management  Processes 

FY 

Nov.  02 

$10,082 

FFRDC  (MITRE) 

FY 

Nov.  02 

$18,070 

$79,070 

Core  Infrastructure  Support  Projects* 

Security  and  Technology  Infrastructure  Releases 

FY 

May  02 

$43,973 

Infrastructure  Shared  Services 

FY 

Nov.  02 

$39,747 

Enterprise  Systems  Management 

FY 

July  02 

$1 1 ,323 

Development  Integration  &  Testing  Environment 

FY 

May  02 

$12,916 

$107,959 

‘Program  Level  and  Core  Infrastructure  Support  Projects  are  funded  on  a  fiscal  year  (FY)  basis  rather  than  by  milestone. 
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Data  Projects 

Customer  Account  Data  Engine  -  MF  R1  MSS 

Dec.  02 

$S,79S 

Customer  Account  Data  Engine  -  IMF  R2  MS4 

Dec.  02 

$38,400 

Customer  Account  Data  Engine  -  IMF  R3  MSS 

June  03 

$9,779 

Enterprise  Data  Warehouse  R3  MS4,5 

May  04 

$4,S00 

Custodial  Accounting  Project  TASL  B1  B2  MS4,5 

Feb.  04 

$36,S00 

Integrated  Financial  Services/  Core  Financial  Systems  MS4 

Nov.  02 

$17,2S0 

$112,224 

Business  Projects 

r 

IR/FoF  R1  MSS 

July  02 

$S,000 

e-Services  2002  MS4,S 

Oct.  03 

$26,092 

Customer  Account  Management  MSS 

Oct.  02 

$24,494 

Filing  &  Payment  Compliance  MS2,3 

Jan.  03 

$14,100 

Reporting  Compliance  -  Individual  MS2,3 

Nov.  02 

$10,000 

HR  Connect  MS4,S 

Dec.  02 

$10,000 

$89,686 

Addition  to  Management  Reserve 

$2,061 

Totai  Business  Systems  Modernization  Program 

$391,000 
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Appendix  II: 
Results  of  Past  GAO  Reviews 


Spending  Plan 

Results  of  GAO  Review 

1“  Spending  Plan 
(May  1999) 

($35  million  request) 

•  The  plan  satisfied  the  legislative  conditions  for  the  use  of  ITIA  funds  and  was 
consistent  with  our  open  recommendations. 

•  The  plan  was  an  appropriate  first  step,  but  the  key  to  success  would  be 
effective  implementation  of  the  plan. 

•  Future  plans  should  specify  progress  against  prior  plan  commitments,  and  the 
next  plan  should  clarily  IRS/contractor  roles  and  responsibilities.  (See  Tax 
Systems  Modernization:  Results  of  Review  of  IRS’  Initial  Expenditure  Plan, 
GAO/AIMD/GGD-99-206,  June  15,  1999) 

I"'  Interim  Spending  Plan 
(Dec  1999) 

($33  million  request) 

•  The  plan  raised  concerns  about  projects  that  were  scheduled  to  begin  detailed 
design  and  software  development  before,  among  other  things,  the  enteprise 
architecture  was  completed  tmd  the  ELC  was  defined  and  implemented. 

•  IRS  should  expedite  completion  of  the  architecture  and  implementation  of  the 
ELC. 

•  Future  plans  should  explain  how  IRS  plans  to  manage  the  risk  of  performing 
detailed  design  or  development  work  if  the  architecture  is  not  sufficiently 
completed  or  the  ELC  is  not  sufficiently  implemented. 
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Results  of  Past  GAO  Reviews 


Spending  Plan 

Results  of  GAO  Review 

2”“  Spending  Plan  (Mar  2000) 
($176  million  request) 

•  IRS  met  relatively  few  commitments  in  its  $35  million  first  ITIA  spending  plan, 
even  though  the  Service  later  received  an  additional  $33  million  and  nearly  5 
months  of  extra  time  to  accomphsh  the  goals  set  forth  in  the  first  plan. 

•  The  plan  satisfied  the  legislative  conditions  for  the  use  of  ITIA  funds,  and  was 
generally  consistent  with  recommendations  contained  in  our  earher  reports. 

•  The  key  to  success  would  be  whether  IRS  effectively  implements  the  plan. 

•  Until  IRS  completes  its  initiated  actions  to  redirect  and  restructure  its 
modernization  effort,  it  would  continue  to  lack  key  modernization  and 
technical  controls.  (See  Tax  Systems  Modernization:  Results  of  Review  of  IRS’ 
March  7,  2000,  Expenditure  Plan,  GAO/AIMD-00-175,  May  24,  2000) 

2°‘  Interim  Spending  Plan 
(Aug  2000) 

($33  million  request) 

•  IRS  had  not  adhered  to  the  approved  and  funded  March  7,  2000,  spending  plan. 

•  On  selected  initiatives,  IRS  had  not  met  cost  and  schedule  commitments  made 
in  its  March  7,  2000  spending  plan. 

•  Most  modernization  initiatives  had  nevertheless  made  important  progress 
since  March  2000.  IRS  fully  addressed  two  of  its  modernization  management 
capability  weaknesses,  and  it  was  making  progress  in  addressing  others. 

•  One  project.  Custodial  Accounting  Project  (CAP),  had  been  approved  for 
product  development  without  sufficient  definition  and  without  a  compelhng 
business  case.  Further  investment  in  CAP  should  be  limited  until  IRS 
demonstrates  sufficient  business  value  and  reports  to  the  House  and  Senate 
committees  on  risk  mitigation. 

•  Another  project.  Security  and  Technology  Infrastructure  Release  (STIR),  was 
being  preliminarily  designed  without  sufficient  requirements  definition  and 
economic  justification.  The  STIR  project  should  be  directed  to  complete  a 
security  risk  assessment  as  soon  as  possible,  and  ensure  that  STIR 
requirements  and  the  proposed  design  solution  are  economically  justified 
through  a  business  case.  (See  Tax  Systems  Modernization:  Results  of  Review 
of  IRS’  August  2000  Interim  Spending  Plan,  GAO-01-91,  Novembers,  2000) 
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Spending  Plan 

Results  of  GAO  Review 

3rd  Spending  Plan  (Oct  2000) 
($200  million  request) 

•  IRS’  plan  satisfied  the  legislative  conditions  for  the  use  of  ITIA  funds,  and  was 
making  important  progress  towards  satisiying  the  congressional  direction  on 
two  projects  -  CAP  and  STIR. 

•  IRS  was  making  important  progress  in  establishing  effective  modernization 
management  capability,  but  important  and  chaUenging  work  remained.  Until 
IRS  completed  its  initiated  actions  to  fuUy  implement  its  system  life  cycle 
methodology  and  business  systems  modernization  office,  and  resolve  issues 
concerning  the  completeness  and  accuracy  of  enterprise  architecture,  it 
continued  to  lack  key  modernization  and  technical  controls. 

•  Five  modernization  initiatives  experienced  schedule  delays  and/or  cost 
increases.  However,  the  third  plan  did  not  address  whether  projects’  prior 
commitments  for  delivery  of  promised  systems  capabilities  (requirements) 
and  benefit/business  value  were  being  met. 

•  IRS  used  contractor-provided  “rough  order-of-magnitude”  estimates  in 
preparing  the  third  expenditure  plan.  IRS  planned  to  validate  the  third  plan’s 
estimates  as  part  of  its  process  to  negotiate  and  definitize  contract  task  orders. 
Previously,  this  process  resulted  in  finalized  contract  costs  below  the 
estimates,  totalling  $9  million.  (See  Tax  Systems  Modernization:  Results  of 
Review  of  IRS’  Third  Expenditure  Plan,  GAO-0 1-227,  January  22,  2001) 
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Spending  Plan 

Results  of  GAO  Review 

4th  Spending  Plan  (March  2001) 
($128  million  request) 

•  IRS’  plan  satisfied  the  conditions  specified  in  the  appropriations  acts. 

•  IRS  continued  to  make  important  progress  in  implementing  modernization 
management  controls  and  capabilities.  Nevertheless,  IRS’  modernization 
management  capacity  is  still  not  where  it  should  be,  given  (1)  the  number  of 
systems  acquisition  projects  that  the  March  2001  plan  identifies  as  underway 
and  planned  and  (2)  the  fact  that  several  of  the  ongoing  projects  are  entering 
critical  stages  in  their  life  cycles.  For  example,  IRS  did  not  have  a  sufficiently 
defined  version  of  the  enterprise  architecture  to  guide  and  constrain  projects, 
and  employing  rigorous  configuration  management  practices. 

•  Due  to  missing  management  capacity,  key  IRS  projects  were  beginning  to 
experience  cost,  schedule,  and  performance  shortfalls  against  the 
commitments  the  agency  made  in  its  third  expenditure  plan.  For  example, 
deployment  of  the  Customer  Communications  2001  project  was  three  months 
behind  schedule,  and  promised  system  capabilities  and  associated  benefits 
had  been  deferred.  Also,  a  critical  infrastructure  project,  STIR,  was  reported 
to  be  1.5  months  late  in  trying  to  complete  its  preliminary  design  phase 
(Milestone  3);  and  the  agency  was  stiU  working  to  finalize  6  of  19  work 
products  needed  to  complete  the  phase.  Thus,  the  project  was  actually  almost 
five  months  late. 

•  IRS  officials  recognized  the  need  to  address  its  modernization  management 
capacity  before  key  ongoing  projects  moved  into  critical  life-cycle  phases,  and 
before  additional  projects  were  started.  Accordingly,  IRS  planned  or  had 
initiated  steps  to  address  these  weaknesses.  In  particular  the  Commissioner 
had  decided  to  slow  ongoing  and  new  projects,  giving  priority  to  putting  in 
place  missing  management  capacity.  We  believed  this  decision  was  prudent 
and  appropriate  and  made  recommendations  to  ensure  IRS  followed  through 
on  this  decision.  (See  Business  Systems  Modernization:  Results  of  Review  of 
IRS’  March  2001  Expenditure  Plan,  GAO-0 1-716,  June  29,  2001) 
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Program/  Project 
Management  Initiative 

Commitment  Date  and 
Funding  as  of  3/2001 
($000) 

Revised  Commitment 
Date  and  Funding 
($000) 

Change  (%) 

Customer 

07/31/01 

09/30/01 

+2  months 

Communications 

$17,787 

$13,696 

-$4,091  (-23%) 

Milestone  3  (release 
2002) 

Customer 

5/31/01 

10/31/01 

+5  months 

Communications 

$43,386 

$45,174 

+$1,788  (4%) 

Milestone  4,  5  (release 
2001) 

e-Services 

6/30/01 

8/14/01 

+  1.5  months 

Milestone  2,3 

$17,879 

$16,819 

-$1,060  (-6%) 

Customer  Account 

11/30/01 

03/31/02 

+4  months 

Management 

$12,135 

$13,100 

+$965  (8%) 

Milestone  2 

Filing  and  Payment 

11/30/01 

01/31/03 

+  14  months 

Compliance 

$2,517 

$3,017 

+500  (20%) 

Milestone  2,3 

Reporting  Compliance- 

11/30/01 

11/30/02 

+  12  months 

Individual 

$6,022 

$10,000 

+3,978  (66%) 

Milestone  2 

Customer  Account  Data 

4/30/01 

6/30/01 

+2  months 

Engine  (CADE) 

$16,567 

$19,267 

+$2,700(16%) 

Milestone  2,3 

Customer  Account  Data 

12/31/01 

6/30/02 

+6  months 

Engine  (CADE) 

$40,038 

$45,338 

+$5,300  (13%) 

Milestone  4 

Custodial  Accounting 

7/31/03 

03/31/03 

-4  months 

Project  (CAP) 

$44,130 

$51,430 

+$7,300  (17%) 

TASLBl  B2 
Milestone  4,5 
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Program/  Project 
Management  Initiative 

Commitment  Date  and 
Funding  as  of  3/2001 
($000) 

Revised  Commitment 
Date  and  Funding 
($000) 

Change  (%) 

Integrated  Financial 
Services/Core  Financial 
Systems 
Milestone  2,3 

3/31/02 

$8,565 

5/31/02 

$8,200 

+2  months 
-$365  (-4%) 

Security  and 
Technology 
Infrastructure  Releases 
(STIR) 

Milestone  4 

9/30/01 

$25,178 

1/31/02 

$31,287 

+4  months 
+$6,109  (24%) 

Virtual  Development 
Environment  (VDE) 

(thru  end  of  FYOl) 
$6,340 

$7,693 

+$1,353  (21%) 

Program  Management 
Office  (formally  PPMO) 

(thru  end  of  FYOl) 
$21,716 

$22,448 

+$732  (3%) 

Program  Management 
Office  (formally  PPMO) 

(1"‘6  weeks  ofFY02) 
$2,236 

$4,124 

+$1,888  (84%) 

Vision  and  Strategy-Tax 
Administration  (TAVS) 
Milestone  1 

$8,376 

$8,996 

+$620  (7%) 

Internal  Management 
Vision  and  Strategy 
(IMVS) 

Milestone  1 

$4,227 

$4,377 

+$150  (4%) 

Configuration 

Management 

(thru  end  of  FYOl) 
$1,687 

$1,885 

+$198  (12%) 

Architectural 
Engineering  Office 
(AEO) 

(thru  end  of  FYOl) 
$19,670 

$19,906 

+$236  (1%) 
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DEPARTMENT  OF  THE  TREASURY 
INTERNAL  REVENUE  SERVICE 
WASHINGTON,  D.C.  20224 


February  15,  2002 


Mr.  Randolph  C.  Hite 
Director,  Information  Technology 
Systems  Issues 
U.S.  General  Accounting  Office 
441  G  Street,  N.W. 

Washington,  D.C.  20548 

Dear  Mr.  Hite: 

I  appreciate  the  GAO's  review  of  the  IRS's  Business  Systems  Modernization  (BSM) 

Plan  for  FY  2002.  The  release  of  the  appropriated  $391  million  based  on  the  GAO's 
review  will  make  it  possible  for  us  to  enhance  services  to  the  American  taxpayer  and 
improve  the  efficiency  of  the  IRS  operations.  I  welcome  the  opportunity  to  respond  to 
the  review  and  address  the  issues  you  raised. 

Modernization  of  the  IRS's  information  systems  and  business  processes  is  one  of  the 
largest  and  most  complex  business  systems  modernization  programs  ever  undertaken. 
Consequently,  we  have  no  clear  precedent  for  charting  our  course.  We  continue  to 
mature  as  an  organization  while  we  undergo  enormous  change  and  simultaneously 
support  the  operations  of  the  agency.  We  have  learned  that  in  any  such  complex 
undertaking,  continuous  changes  will  occur,  and,  therefore,  we  must  adjust  our  plans  as 
we  learn.  Nevertheless,  we  are  committed  to  developing  a  program  that  employs  sound 
management  practices  and  uses  our  resources  wisely. 

We  made  significant  progress  in  FY  2001  in  our  BSM  program.  The  IRS  implemented 
its  first  two  modernized  systems,  Customer  Communications  2001  and  Customer 
Relationship  Management  Examination.  Both  are  delivering  business  benefits  to 
taxpayers  and  the  IRS.  In  addition,  we  made  significant  advancements  on  many  other 
projects  slated  for  FY  2002  delivery  including: 

1 .  Internet  Refund/Fact  of  Filing  (which  will  allow  taxpayers  to  use  the  Internet  to  check 
on  the  status  of  their  refunds), 

2.  Customer  Account  Data  Engine  Release  1  (a  pilot  of  the  first  part  of  the  modernized 
individual  taxpayer  data  base), 

3.  A  production  pilot  of  HR  connect  (a  human  resources  management  system  being 
implemented  Department-wide  by  Treasury),  and 

4.  E-services  (electronic  problem  resolution  and  relationship  management  for  third 
party  practitioners). 
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Further,  the  first  release  of  the  Enterprise  Architecture  was  delivered  last  January,  as 
well  as  a  major  upgrade  in  May.  The  Enterprise  Architecture  inciudes  a  technicai 
architecture,  high-ievei  business  processes  and  an  enterprise  transition  plan  based  on 
Tax  Administration  and  Internal  Management  Visions  and  Strategies.  An  updated 
release  is  in  progress  for  completion  later  this  year. 

We  have  made  important  progress  in  implementing  key  infrastructure  components 
needed  to  support  future  development  and  operation  of  BSM  systems,  such  as  new 
hardware  and  systems  software,  a  robust  testing  environment,  Internet  portals,  security 
infrastructure,  and  systems  and  network  management  capabilities.  We  have  leveraged 
other  resources  within  the  Modernization,  Information  Technology  and  Security  (MITS) 
Services  organization,  such  as  security,  product  assurance,  and  production  operations. 

We  have  also  made  progress  in  Program  management  over  the  past  year.  A  network  of 
Executive  Steering  Committees,  chaired  by  senior  IRS  executives  and  with  broad 
business  participation,  are  key  elements  of  our  BSM  Program  management  structure. 
Project  advisory  councils,  business  process  owners,  our  Enterprise  Life  Cycle 
Methodology,  and  a  network  of  systems  engineering  and  change  control  boards  are  all 
operating  effectively.  In  addition,  we  redesigned  our  management  meetings  to  increase 
communications,  control,  and  address  critical  management  and  integration  issues  on  a 
more  timely  basis. 

The  combination  of  improvement  management  processes  and  a  stronger  executive 
management  team  has  kept  the  Program  under  control.  Further,  as  noted  in  the  GAO 
Report,  we  made  important  progress  in  implementing  key  management  processes,  such 
as  Configuration  Management  and  Risk  Management,  though  we  acknowledge  that 
further  improvements  are  needed  before  these  management  processes  are 
implemented  fully.  We  will  review  with  the  GAO  our  commitment  to  achieving  full 
implementation  as  soon  as  practical. 

We  have  addressed  many  of  your  recommendations,  such  as  prudently  slowing  existing 
projects  and  deferring  new  projects  when  management  capacity  is  inadequate  to 
proceed  with  acceptable  risk. 

GAO'S  Report  cites  18  projects  that  "have  experienced  cost  and/or  schedule  shortfalls 
against  commitments  made  in  the  fourth  and  other  prior  plans."  Many  of  the  variances 
from  plan  are  minor  (less  than  10  percent),  and  several  were  cases  where  actual  costs 
were  less  than  estimated  (the  total  variance  across  all  18  projects  is  under  10  percent). 
Also,  in  many  cases,  these  variations  were  the  result  of  prudent  management  actions 
(fornially  approved  by  either  the  Core  Business  Systems  Executive  Steering  Committee 
or  the  appropriate  Sub-ESC,  and  reviewed  with  GAO  and  0MB).  Examples  include 
deferring  the  Enterprise  Systems  Management  project  to  align  its  testing  schedule  with 
other  products  to  reduce  overall  cost,  and  deferring  the  start  of  Reporting  Compliance 
due  to  lack  of  management  capacity. 
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Many  of  these  commitments  were  simply  planning  estimates  made  early  (prior  to 
Milestone  3  exit)  in  the  project  life  cycle  (ELC).  Our  track  record  demonstrates  effective 
control  of  BSM  projects.  However,  we  agree  with  the  GAO  that  all  management 
processes  must  be  implemented  fully  since  the  complexity  of  the  Program  will  increase 
dramatically  as  we  go  forward.  We  are  neither  complacent  nor  satisfied  with  our 
present  capabilities.  We  will  provide  a  full  explanation  of  the  root  causes  of  each  of  the 
cost  or  schedule  differences  in  a  separate  communication. 

In  addition  to  the  internal  management  controls  that  are  in  place,  those  we  are  adding, 
and  the  legal  controls  that  we  operate  under,  the  BSM  Program  performs  in  an 
exceptionally  open  environment,  under  special  legislative  controls,  and  with  a  rigorous 
and  effective  governance  process.  These  controls,  as  well  as  our  own  management 
judgment,  ensure  that  we  have  adequate  management  capacity  and  an  acceptable  risk 
before  starting  future  work.  The  GAO  has  indicated  that  we  need  to  do  a  better  job  of 
explaining  all  unmet  commitments  in  prior  Plans  in  the  current  expenditure  Plan,  and  we 
will  do  so.  BSM  executives  will  continue  to  hold  weekly  meetings,  as  we  have  done 
throughout  FY  2001 ,  with  the  GAO  to  inform  you  of  events  as  they  occur. 

I  agree  with  your  recommendations  and  have  set  target  plans  and  schedules  to  fully 
implement  Configuration  Management,  Risk  Management,  and  cost  and  schedule 
estimating  processes,  as  well  as  your  recommendations  in  human  capital  management 
and  other  areas.  We  have  reviewed  these  target  dates  with  your  office.  In  addition  to 
the  dedicated  staff  assigned  to  these  improvements,  both  the  IRS  and  PRIME  have 
appointed  senior  executives  to  coordinate  within  and  across  our  respective 
organizations  to  ensure  we  give  top  priority  to  completing  their  implementation. 

Regarding  your  recommendation  to  achieve  the  Software  Engineering  Institute's 
Software  Acquisition  Capability  Maturity  Model  (SA-CMM)  Level  2  certification,  we  have 
made  progress.  We  acknowledge  that  this  has  taken  longer  than  we  expected,  and  our 
past  commitment  to  achieve  this  by  September  2001  was  unrealistic,  based  on  how 
long  other  organizations  are  taking  to  achieve  this  level  of  certification.  I  believe  our 
plan  is  now  reasonable,  we  have  developed  these  plans  working  closely  with  MITRE, 
the  Systems  Engineering  Institute  of  Carnegie  Mellon  University,  and  the  PRIME 
contractor,  who  has  already  achieved  this  certification  level  in  their  Civilian  Government 
Group.^  This  thorough  preparation  and  planning  ensures  that  we  have  a  realistic 
approach  to  achieving  SA-CMM  Level  2  certification  as  targeted. 

Regarding  your  recommendation  to  align  the  pace  of  the  program  with  the  maturity  of 
the  BSM  Organization’s  management  controls  and  management  capacity,  we  agree  and 
are  committed  to  comply.  To  illustrate,  we  are  conducting  a  Portfolio  Reassessment  for 
the  FY  2002  plan,  with  participation  from  IRS's  business  units,  MITRE  and  PRIME.  Our 
Core  Business  Systems  Executive  Steering  Committee  will  approve  this  reassessment. 


'  Only  one  other  government  organization,  the  Abrams  Battle  Tank  Program  in  the  Department  of 

Defense,  has  achieved  SA-CMM  Level  2  certification,  and  they  just  did  so  last  November. 

^  CSC's  Civilian  Group  was  the  first  organization  to  achieve  SA-CMM  Level  2,  and  expects  to  receive  their 
SA-CMM  Level  3  certification  soon. 
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We  recently  completed  a  major  effort  to  "refresh"  the  Tax  Administration  Vision  and 
Strategy  (TAVS),  which  looks  ahead  to  FY  2003  and  beyond,  and  will  "institutionalize" 
the  value  of  this  vital  program  asset.  In  all  these  reviews,  we  look  for  opportunities  to 
make  prudent  adjustments  to  the  program  plan  to  lower  execution  risk.  We  are 
proactively  keeping  all  of  our  stakeholders  informed  of  any  changes  we  make  as  a 
result  of  the  Portfolio  Reassessment  process,  and  we  plan  to  conduct  regular  Portfolio 
Reassessments  in  the  future. 

We  must  maintain  the  momentum  created  in  FY  2001  as  we  move  into  FY  2002  and 
beyond,  both  to  deliver  benefits  to  taxpayers  and  IRS  employees  to  maintain  strong 
buy-in  from  IRS  executives  and  other  key  stakeholders.  Hence,  we  need  to  strike  a 
proper  balance  between  delivering  business  value,  building  critical  infrastructure,  and 
implementing  the  management  processes  necessary  to  ensure  control  and 
effectiveness.  We  believe  our  FY  2002  Expenditure  Plan  strikes  this  balance,  but  will 
continue  to  reexamine  our  commitments  as  per  the  GAO's  recommendations. 

IRS  leadership  understands  the  importance  of  correcting  the  issues  you  identified  and  is 
moving  aggressively  to  resolve  them.  I  appreciate  your  acknowledgement  of  the 
important  progress  we  made.  I  also  appreciate  your  hard  work  in  preparing  your 
comprehensive  report,  your  support  of  the  BSM  Program,  and  your  ongoing  counsel. 

Sincerely, 

<9,  Xjffcro 
Charles  O.  Rossotti 
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GAO’s  Mission 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help  improve 
the  performance  and  accountability  of  the  federal  government  for  the  American 
people.  GAO  examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of  accountability, 
integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  is  through  the 
Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full-text  files  of 
current  reports  and  testimony  and  an  expanding  archive  of  older  products.  The 
Web  site  features  a  search  engine  to  help  you  locate  documents  using  key  words 
and  phrases.  You  can  print  these  documents  in  their  entirety,  including  charts  and 
other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  E-mail 
this  list  to  you  every  afternoon,  go  to  www.gao.gov  and  select  “Subscribe  to  daily 
e-mail  alert  for  newly  released  products”  under  the  GAO  Reports  heading. 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each.  A  check 
or  money  order  should  be  made  out  to  the  Superintendent  of  Documents.  GAO 
also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a  single 
address  are  discounted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

P.O.  Box  37050 

Washington,  D.C.  20013 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

Visit  GAO’s  Document 
Distribution  Center 

GAO  Building 

Room  1100,  700  4th  Street,  NW  (comer  of  4th  and  G  Streets,  NW) 

Washington,  D.C.  20013 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm. 

E-mail:  fraudnet@gao.gov,  or 

1-800-424-5454  or  (202)  512-7470  (automated  answering  system). 

Plllolic  AffrlirS  NelUgan,  Managing  Director,  NelliganJ@gao.gov  (202)  512-4800 
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